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TITLE OF THE INVENTION 

Image-Processing Apparatus and Image-Processing Method 
BACKGROUND OF THE INVENTION 

The present invention relates to an apparatus for processing the image data 
input from an image-handing device and then outputting the image data to another 
image-handling device. The invention also relates to a method of processing image 
data input from an image-handing device before the image data is output to another 

image-handling device. 

As various systems which handle color images come into use in increasing 
nmnbers, it is demanded that color images be reproduced in the same hues by the 
devices of different types that are used in these systems. To meet this demand, image- 
processing apparatuses have been proposed, each designed to evaluate the devices for 
their characteristics and adjust the color values of the image output from the devices 
to the same value. FIG. 1 shows such an image-processing apparatus. FIG. 2 
illustrates the flow of image data in the image-processing apparatus shown in FIG. 1. 

Even if the color values of the images output from the devices are adjusted to 
the same value, however, the colors of the images do not always appear identical to 
human eyes. This is because the observation enviromnent including ambient light 
influences the human visual sense. That is to say, the same color looks different in 
different observation environments. 

The devices may differ in tenns of the darkest points. (That is, the black part 
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of the image that one device outputs may differ from that of the image that another 
device outputs.) Particularly in this case, the images output by the devices appear 
greatly different. Assxome that the darkest point of an input device differs from that 
of an output device. Then, either graying or black-emphasizing occurs in the image 
output by the output device. Consequently, the input image and the output image may 
look quite different in terms of color. 
BRIEF SUMMARY OF THE INVENTION 

The present invention has been made in consideration of the foregoing. The 
object of the invention is to provide an image-processing apparatus and an image- 
processing method which process image data so that the images represented by the 
data and reproduced by different devices may look ahnost identical in tenns of color. 

According to the first aspect of the present invention, there is provided an 
image-processing apparatus for processing the image data input from an image- 
handling device and then outputting the image data to another image-handling device. 
The image-processing apparatus comprises black-adaptation correction means for 
correcting the image data in consideration of the fact that adaptation to black varies 
from person to person, if the darkest points of the image-handling devices differ from 
each other, so that the colors of the images produced by the image-handling devices 
look ahnost the same. 

In the unage-processing apparatus, the black-adaptation correction means 
corrects the image data if the darkest points of the image-handling devices differ from 
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each other, in consideration of the fact that adaptation to black varies from person to 
person. Hence, the colors of the images produced by the iinage-handUng devices look 
almost the same even if the darkest points of the image-handling devices differ from 
each other. 

According to the second aspect of the invention, there is provided a method of 
processing image data input from an image-handing device before the image data is 
output to another image-handling device. The method comprises the step of: 
correcting the image data in consideration of the fact that adaptation to black varies 
from person to person, if the darkest points of the image-handling devices differ from 
each other, so that the colors of the images produced by the image-handling devices 
look almost the same. 

In the image-processing method, the image data is corrected if the darkest 
points of the image-handling devices differ from each other, in consideration of the 
fact that adaptation to black varies from person to person. Hence, the colors of the 
images produced by the image-handling devices look almost the same even if the 
darkest points of the image-handling devices differ from each other. 

As mentioned above, in the present invention, image data is corrected if the 
darkest points of image-handling devices differ from each other, in consideration of 
the fact that adaptation to black varies from person to person. Hence, the colors of the 
images produced by the image-handling devices look almost the saine even if the 
darkest points of the image-handling devices differ from each other. In short, the 
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present invention enables the devices of different types to reproduce color images that 
appears identical in color, although the adaptation to black varies from person to 
person 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 

FIG. 1 is a schematic representation of a conventional image-processing 
apparatus; 

FIG. 2 is a diagram illustrating the flow of image data in the conventional 
image-processing apparatus; 

FIG. 3 is a diagram showing an image-processing apparatus according to the 
present invention; 

FIG. 4 is a diagram illustrating the image-processing sequence perfonned in the 
image-processing apparatus shown in FIG. 3; 

FIG. 5 is a diagram explaining the process of converting an image of the color 
gamut for the input-side device to an image of the color gamut for the output-side 
device, through a device optimal color space; 

FIG. 6 is a graph representing the relationship between the color gamut of the 
medium and the black adaptation rate K^^^; 

1/3 

FIG. 7 is a graph shows the relationship between the Y value (Y^^ ) present 

1/3 

in the device optimal color space and the Y value (Y_, ) present in the color space 
which is to be converted to the device optimal color space; 

FIG. 8 is a graph representing specific examples of the function y^, f(Y^ ); 
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FIG. 9 is a diagram showing another image-processing apparatus according to 
the present invention; 

FIG. 10 is a diagram depicting a menu screen designed to set parameters 
conceming the enviromnent in which an image is observed; 

FIG. 1 1 is a diagram showing another image-processing apparatus according to 
the present invention; 

FIG. 1 2 is a diagram illustrating an image-processing apparatus according to the 
invention^ which is provided in the form of a computer system. 
DETAILED DESCRIPTION OF THE INVENTION 

Embodiments of the present invention will be described in detail, with reference 
to the accompanying drawings. 

In the following description, any image displayed by an image display device 
shall be called soft copy image, and any image printed on a recording medium such as 
a paper sheet shall be called hard copy image. In other words, the media for soft copy 
images are image display devices, whereas the media for hard copy images are 
recording media such as paper sheets. 
1. Structure of the Image-Processing Apparatus 

FIG. 3 shows an image-processing apparatus 1 according to this invention. The 
image-processing apparatus 1 receives the image data representing the image displayed 
by an image display 2 such as a CRT display or a liquid crystal display. The apparatus 
1 processes the image data and outputs the same to a printer 3. 
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In this case, the input-side device is the image display 2, and the output-side 
device is the printer 3. According to the present invention, however, the input- and 
output-side devices are not limited to an image display and a printer, respectively. For 
example, the input-side device may be an image scanner, a camera or the like, and the 
output-side device may be an image display or the like. As shown in FIG. 3, the input- 
and output-side devices are connected directly to the image-processing apparatus 1. 
Instead, these devices may be connected to the apparatus 1 by a network. 

As shown in FIG. 3, the image-processing apparatus 1 comprises a image- 
processing section 1 1, a first input-side sensor 12, a second input-side sensor 13, a first 
output-side sensor 15, and a second output-side sensor 16. The image-processing 
section 11 processes the unage data received from the image display 2. The first 
input-side sensor 12 receives light emitted from the image display 2, thereby 
detecting the reflectance and the like of the screen of the image display 2. The second 
input-side sensor 13 detects the ambient light existing at the time of observing the 
image displayed by the image display 2. The first output-side sensor 15 receives the 
light from a print sheet 14 on which the printer 3 will print the image, thereby 
detecting the total luminance and the like of the print sheet 14, The second output-side 
sensor 16 detects the ambient light existing at the tune of observing the image 
printed on the print sheet 14 by the printer 3. The image-processing section 11 
comprises an input-side converter 2 1 , an input-side observation enviromnent changing 
circuit 22, an image-editing circuit 23, an output-side observation enviromnent 



changing circuit 24, and an output-side converter 25. 

The input-side converter 21 receives the RGB value of the image displayed by 
the image display 2. Using an input-side device profile, the converter 21 converts the 
RGB value to an XYZ value, or a tristimulus value based on the human visual sense. 
The input-side device profile is a file that contains conversion formula or a conversion 
table for converting the RGB value received from the image display 2, to a XYZ value 
based on the human visual sense. The profile has been prepared on the basis of the 
characteristics of the image display 2. The XYZ value is supplied fi-om the input-side 
converter 21 to the input-side observation enviromnent changing circuit 22. 

The input-side observation enviromnent changing circuit 22 a conversion 
process based on a chromatic adaptation model (S-LMS) later described, on the XYZ 
value received fi-om the input-side converter 21. Further, the circuit 22 performs a 
conversion process on the XYZ value so that the value may be used in the device 
optimal color space as will be described later. Thus, the circuit 22 converts the XYZ 
value to an XYZ value (X^pY^pZ^^) for use in a device optimal color space. 
(Hereinafter, the XYZ value obtained by the conversion will be called XopYopZo? 
value. The XopYopZop value is supplied fi-om the input-side observation environment 
changing circuit 22 to the image-editing circuit 23. 

To convert the XYZ value received from the input-side converter 2 1 , the input- 
side observation enviromnent changing circuit 22 receives, from the first input-side 
sensor 12, the detection signal representing the reflectance and the like of the screen 
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of the image display 2. The circuit 22 also receives, from the second input-side sensor 
13, the detection signal representing the ambient light that exists at the time of 
observing the image displayed by the image display 2. From these detection signals 
the circuit 22 obtains the parameters concerning the environment in which the image 
displayed by the image display 2 is observed. Using the parameters, the circuit 22 
perfoms a conversion process based on the chromatic adaptation model and a 
conversion process for making it possible to use the XYZ value in the device optimal 
color space. 

The image-editing circuit 23 effects an image-editing process, such as color 
gamut compression, on the X^pY^^Z^^ value received from the input-side observation 
enviromnent changing circuit 22. The X^pY^p^^p value, thus processed, is supplied 
from the image-editing circuit 23 to the output-side observation enviromnent changing 
circuit 24, 

The output-side observation environment changing circuit 24 processes the 
X^pY^pZ^p value, thereby converting the same from one for use in the device optunum 
color space to one for use ia an LMS color space. Further, the circuit 24 perfonns an 
inverse conversion process based on the chromatic adaptation model, which will be 
described later. Thus, the circuit 24 converts the X Y^ Z value received from the 

' OP op OP 

image-editing circuit 23, to an XYZ value, which is a tristimulus value, based on the 
human visual sense. 

In order to convert the X^pY^pZ^^ value received from the image-editing circuit 
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23, the output-side observation environment changing circuit 24 receives the detection 
signal output from the first output-side sensor 15 and the detection signal output from 
the second output-side sensor 16. The detection signal output by the sensor 15 
represents the total Imninance and the like of the print sheet 14 on which the printer 
3 will print the image. The detection signal output by the sensor 16 represents the 
ambient light existing at the time of observing the image printed on the print sheet 
14 by the printer 3. From these detection signals the circuit 24 obtains parameters 
conceming the enviromnent in which the image printed on the print sheet 14 by the 
printer 3 is observed. Using these parameters, the circuit 24 converts the X^pY^pZ^^ 
value, from one for use in the device optimimi color space to one for use in an LMS 
color space, and effects conversion process based on the chromatic adaptation model 
The output-side converter 25 receives the XYZ value from the output-side 
observation enviromnent changing circuit 24. The output-side converter 25 converts 
the XYZ value to a CMY value (or a CMYK value) that the printer 3 will used to print 
the image, by using the an output-side device profile. The output-side device profile 
is a file that contains conversion fonnula or a conversion table for converting the XYZ 
value based on the human visual sense to a CMY value, which the printer 3 will use 
to print the hnage. The output-side profile has been prepared on the basis of the 
characteristics of the printer 3. The CMY value is supplied from the output- side 
converter 25 to the printer 3. The printer 3 prints the image on a print sheet 14. 
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2, Sequence of Proces sin g Image Data 

The above-mentioned sequence of processing image data will be explained, with 
reference to FIG. 4. 

As shown in FIG. 4, the input-side converter 2 1 perfonns a conversion process 
based on the input-side device profile, on the RGB value of the image displayed by the 
image display 2, The RGB value is thereby converted to an XYZ value. 

Next, the input-side observation enviromnent changing circuit 22 subjects the 
XYZ value to a conversion process based on the chromatic adaptation model. The 
XYZ value is thereby converted to an LMS value for used in an LMS color space that 
does not depend on the observation enviromnent. (Hereinafter, the LMS value will be 
called LgMgSg value.) 

The input-side observation environment changing circuit 22 converts the 
L^M^S^ value to an X ^Y^ Z^^ value for use in the device optical color space. 

S S S OP OP op ^ ^ 

Next, the image-editing circuit 23 subjects the Xqp'^op^op ^^^^^ ^ image- 
editing process, such as color gamut compression. In the image-editing process, the 

X^ Y^ Z^^ value is converted to an L*a*b* value for use in a sense-equivalent color 

OP OP op ^ 

space. (Hereinafter, the L*a*b* value will be referred to as Lg*ag'*'bg* value.) The 
circuit 23 then performs the image-editing process, such as color gamut compression, 
on the Lg*ag*bg* value. Further, the circuit 23 converts the Lg*ag*bg* value, thus 
processed, to the XopYopZop value for use in the device optimal color space. 

Then, the output-side observation enviromnent changing circuit 24 converts the 
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X^pYopZop ^^^^^ to a LsM^Sg value for use in the LMS color space that does not 
depend on the observation environment. 

Further, the circuit 24 subjects the L^M^S^ value to a conversion process based 
on the chromatic adaptation model, thus converting the L^M^S^ value to an XYZ value. 

Finally, the output-side converter 25 converts the XYZ value to a CMY value. 
The CMY value is output to the printer 3. 

In the sequence of processing unage data, described above, the data converted 
on the basis of the input-side device profile remains not depending on the color space 
of the device until it is subjected to the conversion process based on the chromatic 
adaptation model. The data converted on the basis of the data converted on the basis 
of the input-side device profile remains not depending on the color space of the device 
until it is subjected to the conversion process based on the chromatic adaptation model 
does not depend on the observation enviromnent until it is subjected to the inverse 
conversion process based on chromatic adaptation model. The data converted for use 
in the device optical color space remains not depending on the dynamic range of the 
device until it is converted to an L^M^S^ value for use in the LMS color space that 
does not depend on the observation enviromnent. 
3. Conversion Process Based on Color Adaptation Model 

The conversion process, which the input-side observation enviromnent changing 
circuit 22 perfonns in the image-processing apparatus 1 on the basis of the chromatic 
adaptation model, will be explained in detail. It should be noted that the output-side 
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observation environment changing circuit 24 effects an inverse conversion process 
based on the chromatic adaptation model, as will be described below. 

In an observation enviromnent including ambient light, the color of the image 
appears differently to human eyes, because the ambient light influences the human 
eyes. Hence, a conversion process based on the chromatic adaptation model is carried 
out to compensate for the change in color of the image viewed in the observation 
enviromnent that includes ambient light. In other words, the conversion process based 
on the chromatic adaptation model is a process of compensating for correcting the 
color of the image viewed in the observation enviromnent that includes ambient light. 

The chromatic adaptation model used here is fundamentally a Von Kries 
adaptation model. The model is processed in three major steps. The first step is 
correction of contrast, the second step is conversion of a tristimulus value to a cone 
signal, and the third step is correction of chromatic adaptation. Of these steps, the 
third is the most important. The third steps consists of two sub-steps. The first sub- 
step is consideration of incomplete adaptation, and the second sub-step is 
consideration of mixed adaptation. These steps of processing the chromatic adaptation 
will be explained below, one after another. 

In the conversion process based on the chromatic adaptation model and the 
conversion process for obtaining values for use in the device optimal color space, both 
will be described later, some parameters are applied, which concern with the 
observation environment in which the image is looked at. These parameters are 
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obtained from the detection signals generated by the first input-side sensor 12, the 
second input-side sensor 1 3, first output-side sensor 1 5 , and second output-side sensor 
16. 

More specifically, in the conversion process perfoniied by the input-side 
observation environment changing circuit 22, the parameters concerning the 
enviromnent in which the image is observed are obtained from the reflectance and the 
like of the screen of the image display 2, which have been detected by the first input- 
side sensor 12, and the ambient light which exists at the time of observing the image 
displayed by the image display 2 and which has been detected by the second input-side 
sensor 13. Similarly, in the conversion process effected by the output-side observation 
enviromnent changing circuit 24, the paratneters concerning the enviromnent in which 
the image is observed are obtained from the total luminance and the like of the print 
sheet 14, which have been detected by the first output-side sensor 1 5, and the ambient 
light which exists at the time of observing the image printed on the print sheet 14 
by the printer 3 and which has been detected by the second output-side sensor 16. 
( 1 ) Correction of Contrast 

Correction of contrast is first performed in the conversion process based on the 
chromatic adaptation model. 

If the ambient light has high luminance, the image that the image display 2 
displays will have low contrast. This is because the screen of the image display 2 
reflects the ambient light, inevitably causing black-emphasis. Most image displays, 
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such as CRT displays and liquid crystal displays, have an antireflection fihn on the 
screen. The antireflection film cannot completely prevent the reflection of ambient 
light at the screen. As long as the ambient light exists, the black reproduced on the 
screen cannot be darker than the black provided by die hght reflected from the screen. 
As the CIELAB fomiula teaches, the human visual sense is keen to dark colors, and 
the contrast of the image decreases if black is emphasized. Therefore, the light 
reflected from the screen of the image display 2 is taken into consideration to 
accomplish the correction of contrast. 

First, an offset value corresponding to the light reflected from the screen is 
added to the XYZ value that has been obtained by converting the RGB value of the 
image displayed by the image display 2. An X' Y' Z' value is thereby obtained as can 
be understood from the following equation ( 1 - 1 ). In the equation ( 1 - 1 ), the XYZ value 
is normalized with Y\,p^ expressed by the following equation(l-2) , so that Y' may 
acquire the maximum value of 1. In the equations (1-1) and (1-2), R^^ is the 
reflectance of the screen of the unage display (usually, 3 to 5% in the case of CRT 
displays), XpR^Ypj^^Zpi^D is the absolute Imninance that the ambient light has when 
reflected by a perfect reflecting diffiiser, and Yj^.^^ is the absolute Imninance of the 
white point of the medimn (i.e., the image display 2). 



14 



T 



X 




Xprd 


Y 


+ Rbk. 


Yprd 


Z 




Zprd 



....(1-1) 



where 

Since the input-side device is the image display 2, it is desirable to perform the 
correction of contrast as has been described above. If the input-side device is an image 
scanner or the like and if the image subjected to the correction of contrast is a hard 
copy image, it will be unnecessary to carry out the correction of contrast. In such a 

case, the X ' 7 ' Z ' value is normalized with Y\f^v expressed by the following 

equation (1-4), thereby obtaining X ' 7 ' Z ' . In the equation (1-4), Yp^p^j. is the 

absolute luminance at the white point of the medium. In the case where an image 
scanner reads the image printed on a paper sheet, Yp^p^j. is the absolute luminance of 
the paper sheet on which the image has been printed. 
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(2) Conversion of Tristimulus Value fXYZ) to Cone Signal 

Next, the tristimulus value (i.e. ^ XYZ value) is converted to a cone signal (LMS 
value). This conversion process uses a matrix. The matrix formulae that can be 
applied to the conversion process are as follows. 

(i) Hunt-Pointer-Estevez Conversion 
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(ii) Bradford Conversion 
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(iii) sRGB Conversion 
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The symbol for the sRGB conversion should be sRGB. Nonetheless, LMS is 
used in the equation (1-7) in order to harmonize the equation (1-7) with the other 
equations (1-5) and (1-6). 

To convert the tristimulus value (XYZ value) to a cone signal (LMS value), the 
most appropriate matrix fonnula is selected from among the above-mentioned 
fonnulae in accordance with the characteristics and the like of the image, and X' Y' 
Z' are converted to LMS in accordance with the matrix formula selected. 
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The conversion of the tristimulus value (XYZ value) to the cone signal (LMS 
value) is not absolutely necessary. Rather, this conversion need not be carried out. 
If the conversion is not effected, the symbol of X' Y' T should be used in the 
equations that will follow. Instead, LMS will be used in the following equations for 
hanuordzation of description, even if this conversion is not perfonued. 

(3) Correction of Chromatic Adaptation 

Next, the correction of chromatic adaptation is performed in accordance with 
the enviromnent in which the image is observed. 

The receptor cells called ones in the hmnan retina have their sensitivity changed 
to perceive a light source as being white, in the same manner as white balance is 
attained in a video camera. That is, the signals generated by the cones are nomialized 
with the value of the white point. To cope with the changes in the sensitivity of the 
cones, the chromatic adaptation is corrected on the basis of the Von Kries adaptation 
model. 

The hmnan visual sense is not always fully adapted to the white point of a light 
source. Hence, the chromaticity of the light is not used as the while point (hereinafter 
referred to as adaptation white point to which the human visual sense is adapted. 
Rather, in the present model, the adaptation white point is detennined from incomplete 
adaptation and mixed adaptation. 
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(3.1) Incomplete Adaptation 

When we look at the image displayed by the image display 2, our visual sense 
tries to adapt itself to the white point of the screen. Even if we observe the image in 
a dark room, the visual sense cannot be perfectly adapted to the white point if the 
white point greatly deviates from the standard illuminating light D^^. Particularly, the 
more the chromaticity of the white point deviates from that of the standard 
illmninating light D^^ or the standard illmTiinating light E, the more incomplete the 
adaptation is. In addition, the lower the luminance of the white point is, the more 
incomplete the adaptation is. In view of the incomplete adaptation, an adaptation 
white point (hereinafter called incomplete adaptation white point is deteraiined in the 
image-processing apparatus 1, 

The incomplete adaptation white point can be determined by, for example, the 
Hunt, R-LAB system, the Naya system and the CIECAM97s system. These systems 
will be described below. In the image-processing apparatus 1, the system, which 
seems the most appropriate in accordance with the characteristics and the like of the 
unage, is selected from these systems, and the incomplete adaptation white point is 
deteraiined by the system selected. 

In the following equations, ^ 'mw^'mw^' \y[\v is the LMS value of the 

incomplete adaptation white point, and L ^ mv is the LMS value obtained 

by normalizing the tristimulus value Xj^j^ Y^^r Z^i^ of the absolute Imtiinance of the 
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white point on the screen. The symbol Y\,^^v is the absolute luminance [cd/m^] 
represented by the equation (1-2) or (1-4) described above, 
(i) Hunt, R-LAB system 
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wherein 



PL=(l+r Mw"^ + fe) / (1 + r m^"^ + 1 / fe) 
Pm = (1 + r Mw"^ + mE) / (1 + r m'"^ + 1 / w^) 

Ps=i\+ y w''^ + / (1 + r w^'^ ^xise) 



.(1-9) 



Ie = 3' Lmt / (Lw + + Smiv) 
mE = 3' Mi\m' I (Lmjv + Maw + S'w) 



...(1-10) 
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(ii) Naya System 
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where 



Fl = Pixk 

P' M = Pm X k 

Fs = Psxk 



k = 0.3710(Iw / Pl) + 0.629 1(Mmt/ / Pv/) 



.(1-12) 



Px = (1 + 7' w"' + fe) / (1 + r w"' + 1 / fe) 
Pm = (1 + F' -w'" + W£) / (1 + r iw'" + 1 / mE) 
Ps=(l+ T Mw'" + se) / (1 + 7' + 1 / 5£) 



...(1-13) 



Ie-3- Lmw I (Lmv + Mm' + Smw) 
mE= 3 • Mmjf / ( Lwv + Mw + Sm^^ 
SE - 3- Smw I {Lmw + + Sa'/jf) 



.(1-14) 
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(iii) System Adopting D Factor Used in CIECAM97s 

Lmr 



L' MW = 

Mm? = 



D+ Lmv(l~ D) 
Maw 



Asr = 



D+ Mmr{\-D) 
D+&mi\-D) 



...(1-14) 



D=F-F/[l+ 2{La'' ) + (L/ / 300)] 
F = const. 



...(1-15) 



In the equation (1-15) presented above, F is a constant that depends on the 
observation environinent. For example, F = 1.9 if the observation environment is 
ordinarily luminous; F = 0.9 if the enviromnent is a dim, with a httle amount of 
ambient Hght; and F =^ 0.9 if the image is printed on a transparent paper sheet. A 

nonlinear exponential parameter needs to be provided for S the actual 

CIECAM97s system,, but such an exponential parameter is deleted from the equation 
(1-14) for the sake of simplicity. The exponential parameter may of course be taken 
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into consideration. 

If the image display 2 is the input-side device, it is desirable to apply incomplete 
adaptation. The input-side device may be an image scanner or the like, and the unage 
processed may be a hard copy image. In this case, it is unnecessary to apply 

incomplete adaptation, and the value of L M S \^^^r (LMS value), obtained by 

nonnalizing the tristimulus value L ^i^^ M^^^^r S ^^^r of the white point of the medimn, 
is applied without being modified, as can be seen from the following equation (1-16). 







Mm' 


-(1-16) 











(3.2^ Mixed Adaptation 

The image displayed by the image display 2, i.e., a soft copy image, is scarcely 
observed in a dark room. In most offices, the image is observed under fluorescent 
lamps that have a correlation color temperature (CCT) of about 4 1 5 OK, On the other 
hand, the correlation color temperature of the white point of CRT displays widely as 
image displays is approximately 9300K. In the case of the soft copy image, the white 
point of the medimn (i.e., image display 2) usually has a color temperature that differs 
fi:*om the color temperature of the ambient light. 

In the case of the image printed on a recording medium (i.e., a hard copy 
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image), the recording medimn is usually a white paper sheet. The image may be 
printed on a yellowish paper sheet, such as newspaper sheets. If the recording medium 
is not perfectly white, the white point of the recording medium differs from the white 
of the ambient light. That is, in the case of a hard copy image, too, the white point of 
the recording medivmi (i.e., a paper sheet or the like) may differ from the color 
temperature of the ambient light in some cases. 

In both the soft copy image and the hard copy image, the white point of the 
medimn may differ from the color temperature of the ambient light. If so, the human 
visual sense is regarded as partly adapted to both the white point and the color 
temperature. Let us assmne, therefore, that the white point to which our visual sense 
is adapted lies some where between the white point of the medimn and the color 

temperature of the ambient light Then, the LMS value (i.e., L M'\^^rS "Mw)-of 
the white point to which our visual sense is actually adapted is defined by the 
following equations (1-17) and (1-18), where R^dp is the adaptation ratio at which the 
human visual sense is adapted to the white of the medium. 
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..(1-17) 



7«.. = • F' M.'- + (1 - Radp) • F }^ . . (1 - 1 8) 

j/1 1Q^ 7 Xf is the LSM value that 

In the equations (1-17) and (1-18), L pro prd prd 

has been obtained by nonnaUzing the tristimulus value X^^^ Y^.^ Z^^ of the absolute 
luminance of the ambient Hght reflected from a perfect reflecting diffuser. Symbol 
YV,^ is the absolute luminance of the white point of the medium. (The absolute 
luminance is one evaluated in consideration of the light reflected from the screen, if 
the medium is the image display 2). If the medium is the image display 2, Y'mw is the 
absolute luminance of the white point of the screen of the display 2. If the medium is 
a white paper sheet, Y\,^ is the absolute luminance of the paper sheet. 

hitroduced into the equations (1-17) and (1-18) are weighting coefficients 
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(Y'jq^r/Yadp)^^^ and (Y'pj^/Y^^p)^'^. These coefficients are applied when the absolute 
luminance of the ambient light reflected fi*om the perfect reflecting diffuser differs 
from the absolute luminance of the white point of the medium. 

In the equations (1-17) and (1-18) presented above, it is desired that the 
adaptation ratio R^^^ be about 0.4 to 0.7 in normal environment. More precisely, is set 
to 0.6. The adaptation ratio R^^^ of 1 .0 means that the human visual sense is perfectly 
(100%) adapted to the medium, not influenced by the ambient light at all In other 
words, the ratio R^^ is 1 .0 when the human visual sense is adapted 100% to the image 
display 2, not influenced by the ambient light, if the medimn used is the image display 
2, or when the visual sense is adapted 100% to the recording medium, not influenced 
by the ambient light, if the recording medium used is, for example, a paper sheet. 
Conceptually, the ratio R^^^ of 1 .0 is achieved when CIELAB is adjusted to the human 
visual sense. The value of 0.0 is attained when the human visual sense is adapted 
100% to the ambient light. This is equivalent to the CIE/XYZ adjusted to the htunan 
visual sense. 

(3.3) Correction of Color Adaptation, Based on Von Kries Model 

The adaptation white point ( L '\^r M'\^i-s?^S'\,^l) that has been obtained in 

consideration of the incomplete adaptation and mixed adaptation is substituted in the 
Von Kries adaptation mle, as indicated in the following equation (1-19). The color 
adaptation is thereby corrected, providing an LMS value (LgMgSg) that does not 
depend on the observation environment. In the equation (1-19), LMS in the right side 
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is a value obtained in the second step of converting a tristimulus value to a cone signal, 
which has been described above. 
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The input-side observation environment changing circuit 22 of the image- 
processing apparatus 1 performs the conversion process based on the chromatic 
adaptation model. Thus, the circuit 22 converts the XYZ value received from the 
input-side converter 21, to an LMS value (LgM^Sg) for the LMS color space that does 
not depend on the observation enviromnent. The output-side observation enviromnent 
changing circuit 24 of the image-processing apparatus 1 effects the inverse conversion 
process based on the chromatic adaptation model. 
4. Conversion Process to the Device Optical Color Space 

The conversion process to the device optimal color space, perfonned by the 
input-side observation environment changing circuit 22, will be described in detail. 

The conversion process to the device optimal color space is carried out in 
consideration of black-adaptation correction. The black-adaptation correction is 
effected so that a color looks almost the same on whichever medium, even if the 
darkest points of the media have different tristimulus values. Suppose an image is 
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merely converted from the color gamut of a medium to the color gamut of another 
medium. Then, the color of the image appears different to the hmnan eyes if the 
darkest points of the media have different tristimulus values. This is inevitably 
because the adaptation to black varies from person to person. This is why the black- 
adaptation correction is perfonned, whereby the color looks almost the same on 
whichever medium even if the darkest points of these media have different tristimulus 
values. 

As shown in FIG. 5, the color space is changed to the device optimal color 
space, thus effecting the black-adaptation correction, in order to converts the image 
from the color gamut of the input-side device to the color gamut of the output-side 
device. Ambient light, if any^ influences the human visual sense. Hence, the color 
gamut S 1 the image has when ambient hght exists differs from the color gamut S2 the 
image has when no ambient light exists. 

As will be explained below, the black-adaptation correction is carried out by 
using the XYZ value of the darkest point on the medium (i.e., the XYZ value of the 
black point) and the XYZ value of the most luirdnous point on the medium (i.e., the 
XYZ value of the white point). 

The conversion process to the device optimal color space is performed in 
consideration of black-adaptation correction, in the following manner. First, the LMS 
value (LgMgSg) defined by the equation ( 1 - 1 9) is converted to a value (X^Y^Z^) for use 
in the XYZ color space, as follows: 
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The XYZ value (X^ ^Y^ J) of the black point that man may perceive as 

S.K. S-K. S.K 



being darkest is defined by the following equation (2-2). This if the black-adaptation 
correction. 



Ysy = Kadp ■ Ys, mk'" + (1 - Kadp) • Ypk"' 

Zs,k"' = Kadp- Zs,J.Ik"' + (1- Kadp)-ZPK"' 



...(2-2) 



In the equation (2-2), X , , ^ ^ is the tristimulus value for the black 

^ V S^UK S,MK S,MK 

point of the medium and X^^^Y^^Z^^^ is the tristimulus value for the light reflected from 

PK. PIC PK. 

the perfectly black part. These values must be 0 in ideal condition. Symbol K^^^ is 
the rate of adaptation to the human eyes and ranges from 0 to 1. FIG. 6 shows the 
relationship between the color gamut of the medium and the rate K^^^ of adaptation to 
black. The adaptation rate K^^^ having this relationship with the color gamut of the 
medium is applied, accomplishing the black-adaptation correction. 

As mentioned above, X^^ Y^^Z^^ is the tristimulus value for the light reflected 

PIC PiC PiC 

from the perfectly black part and must be 0 in the ideal condition. Hence, we can have 
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the following equation (2-3): 

Xpk= Ypk=Zpk= 0...(2- 3) 

Therefore, the equation (2-2) reduces to the following equation (2-4): 



= Kadp-XS,MK'' 
= Kadp-YS,MK"' 



.(2-4) 



Next, the XYZ value (X^ ^Y^ TL^ ^) of the black point, which has been 

S,K. S5K. SjK 

subjected to the black-adaptation correction, is applied to define the device optimal 
color space that is a color space corresponding to the dynamic range of the device. 
More specifically, the device opthnal color space is defined as illustrated by the 
following equation (2-5). That is, the XYZ value (X^ ^Y^ JL^ ^) of the black point, 
which has been subjected to the black-adaptation correction, is combined with the 
XYZ value (X =^ Y = Z = 1) for the adaptation white point and is thereby defined as 
an exponential function. In accordance with the equation (2-5) the XYZ value 
(XgYgZg) obtained by the process of the equation (2-1) is converted to an XYZ value 
(X^pY^pZ^p) in the device optimal color space that corresponds to the dynamic range 
of the device. 
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FIG. 7 is a graph showing an example of the function expressed by the above 
equation (2-5). In FIG. 7, the Y value (Y^^^ present in the device optimal color 

1/3 

Space is plotted on the ordinate, and the Y value (Y^ ) present in the color space to 
be converted to the device optimal color space is plotted on the abscissa. As FIG. 7 
shows, the device optimal color space is only a fraction of the dynainic range if K^^^ 
= 0 (that is, if no black-adaptation correction is carried out). This space is one much 
contracted. If K^^p is greater than 0 (K^^^ < 0), the device optunal color space expands. 
In particular, the device optimal color space becomes a space fully corresponding to 
the black adaptation when K^^^ =^ 1.0; it becomes a space that utilizes the entire 
dynamic range of the device. 

The exponential parameters y^, and y^ in the equation (2-5) are functions of 
K is shown in the following equations (2*6): 

31 



Yz = f{Zs,K) 



...(2-6) 



The results of the experiment the inventor conducted show that these exponential 
parameters y^, and y^ should better be 1 when and Zg is 0, and should 

better be greater than 1 and undergo simple increase when ^ and Z is greater 

than 0. That is, it is desired that these parameters y^, y^ and y^ be defined as functions, 
each assuming the value of 1 when X^ Y^ ^ and ^ is 0, and assuming the value 
greater than 1 and simply increasing when X_^ „ Y_, ^ and ^ is greater than 0. 
Specific examples of such a function are shown in FIG. 8. More precisely, FIG. 8 is 
a graph representing two examples Al and A2 of the function f of the exponential 
parameter, i.e., y^ = f (Y^^). 

In the image-processing apparatus 1, the input-side observation environment 
changing circuit 22 performs the conversion process to the device optimal color space 
in consideration of the black-adaptation correction. Thus, the LMS value (LgM^Sg) 
obtained in the conversion process based on the color adaptation model is converted 
to an XYZ value (X^pY^pZ^p for use in the device optimal color space that 
corresponds to the dynamic range of the device, 
5. Process after Conversion to Device Optimal Color Space 
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The XYZ value (X^pY^^Z^p) obtained in the conversion process to the device 
optimal color space is supplied to the image-editing circuit 23. The image-editing 
circuit 23 perfonns an image-editing process, such as color gamut compression. In the 
UBage-editing process, the X^pY^^Z^^ value is converted to an Lg^ag^bg"*" value. This 
conversion is expressed by the following equations (2-7): 



Z^*= 116'(7opf''16 

a/ = 500- ((Xoj^y'' .(Fopf'} 

hs^ = tan- 1(-^) 



..(2-7) 



Then, the hnage-editing circuit 23 perfomis an image-editing process, such as 
color gamut compression, on the Lg^a^^b^* value thus obtained. Thereafter, the 
image-editing circuit 23 converts the L *a„*b^* value to the XYZ value. The 
'^op^op'^op ^^^^ supplied to the output-side observation enviromnent changing 
circuit 24. The iinage-editing process is not absolutely necessary in the present 
invention; it may not be carried out. 
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The ^opY^pZ^p value is supplied to from the output-side observation 
environment changing circuit 24 to the output-side device through the output-side 
converter 25. The output-side converter 25 needs to have a color space optimal to it. 
This is because the X^^Y^pZ^^ value must pass through an optimal color space before 
supplied to the output-side device. The optimal color space of the output-side device 
is identical to the optimal color space of the input-side device. Therefore, the 
following equations (2-8) hold true. Suffix IN to some symbols in the equations (2-8) 
et. seq. indicates that the values pertain to the input-side device, while suffix OUT to 
some other symbols means that the values pertain to the output-side device. 



(Zout^opY^^ ~ (Zin^opY^^ 



.(2-8) 



The device optimal color spaces can be given by the equations (2-5) shown 
above. The equations (2-5) can be replaced by the following equations (2-9): 
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From the equations (2-10) we can find the X^Y^Z^ value for the output side. 
The inverse conversion represented by the equation (2-1) is effected, converting the 

Y value to the L M S„ value for use in the LMS color space that does not depend 

s s s s s s 

on the observation environment. Thereafter, the inverse conversion process based on 
the color adaptation model is performed on the LgM^S^ value, thereby converting the 
L^M S value to the XYZ value. The output-side observation enviromnent changing 
circuit 24 supplies the XYZ value to the output-side converter 25. 

The output-side converter 25 converts the XYZ value supplied from the circuit 
24, to a CMY value on the basis of the output-side device profile. The output-side 
converter 25 outputs the CMY value to the printer 3. 

As described above, the black-adaptation correction is effected the adaptation 
to black varies from person to person. As a result, the colors of the image appears 
almost the same in both the input-side device and the output-side device, even if the 
darkest points of the input- and output-side devices differ from each other. Thus, in 
the image-processing apparatus 1 described above, the image displayed by the image 
display 2 and the image printed by the printer 3 which has received the data 
representing the image produced by the display 2 looks ahnost the same, though the 
darkest points of the image display 2 and the printer 3 differ from each other. 
6 . Other Embodiments 

In the embodiment shown in FIG. 3, the first and second input-side sensors 12 
and 13 and the first and second output-side sensors 15 and 16 are used to obtain the 
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parameters concerning the observation environment, which are applied in the image- 
converting process. These parameters may be input directly to the image-processing 
apparatus 1. 

FIG. 9 shows an image-processing apparatus 51 that directly receives the 
parameters concerning the enviromnent in which the image is observed. The 
components of the apparatus 51 shown in FIG. 9, which are similar to those of the 
image-processing apparatus 1 shovm in FIG. 3 are designated at the same reference 
numerals. 

As is illustrated in FIG. 9, the image-processing apparatus 5 1 comprises two 
parameter- setting circuits 52 and 53, in place of the sensors 12, 13, 15 and 16 used in 
the image-processing apparatus 1 of FIG. 3. The parameter-setting circuit 52 is 
connected to the input-side observation enviromnent changing circuit 22, and the 
parameter-setting circuit 53 to the output-side observation enviromnent changing 
circuit 24. 

In the image-processing apparatus 51, the parameters that the circuit 22 needs 
to convert unage data are input to the circuit 22 via the parameter-setting circuit 52 
connected to the input-side observation environment changing circuit 22. Using the 
parameters thus input, the input-side observation environment changing circuit 22 
perfonns the conversion of image data. Further, the parameters that the circuit 24 
needs to convert image data are input to the circuit 24 via the parameter-setting circuit 
53 connected to the output-side observation enviromnent changing circuit 24. Using 
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these parameters, the output-side observation environment changing circuit 24 
performs the conversion of image data. 

To input the parameters concerning the observation enviromnent from the 
parameter-setting circuits 52 and 53, it is desired that a menu screen be used, which 
is, for example, such a graphical user interface as is illustrated in FIG. 10, 

The screen menu shown in FIG. 10 is one used to cause the parameter-setting 
circuit 52 to set parameters. The screen menu is designed, enabling the user of the 
apparatus 51 to select an indoor-light chromaticity (light source), an indoor-light 
luminance (surround luminance) and a Imninance for the image display 2 (monitor 
Ivmiinance), each from a plurality of choices. In the menu screen shown in FIG. 10, 
"F6", "Dark" and "Mid" are selected for the indoor-light chromaticity, indoor-light 
luminance and luminance for the image display 2, respectively. 

The parameter-setting circuit 52 stores the parameters about the observation 
enviromnent which correspond to these various choices. The parameter-setting circuit 
52 reads the parameters that correspond to the choices selected and supplies these 
parameters to the input-side observation environment changing circuit 22. In the case 
of the menu screen shown in FIG. 10, the parameters corresponding to the "F6", 
"Dark" and "Mid" are selected for the indoor-light chromaticity 6, the indoor-light 
luminance "Dark", and the image display luminance "Mid" are supplied to the input- 
side observation enviromnent changing circuit 22. The circuit 22 converts the iinage 
data in accordance with these parameters. 
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In the iitiage-processmg apparatus 1 of FIG. 3, the input-side device and the 
output-side device are the image display 2 and the printer 3, respectively. The input- 
and output-side devices are not limited to these. For example, the input-side device 
and the output-side device may be an image scanner 62 and an image display 63, as in 
the image-processing apparatus 61 illustrated in FIG. 11. The components of this 
apparatus 61, which are similar to those of the apparatus 1 shown in FIG. 3, are 
designated at the same reference numerals as in FIG. 3. 

In the image-processing apparatus 61 of FIG. 11, the first input-side sensor 12 
detects the light reflected from the paper sheet 64 on which the image has been 
printed to be read by the image scanner 62, From the light the sensor 12 determines 
the total luminance of the printed paper sheet 64. The second input-side sensor 13 
detects the ambient light existing when the image printed on the paper sheet 64 is 
observed. The first output-side sensor 15 detects the light from the image display 
63 and determines therefrom the reflectance and the like of the screen of the image 

display 63, The second output-side sensor 16 detects the ambient light L existing 

ft 

when the image displayed by the image display 63 is observed. 

The present invention is not to the embodiments depicted in FIGS. 3 and 11. 
Rather, the invention can be applied to such a computer system 71 as is illustrated in 
FIG. 12. 

The computer system 71 of FIG. 12 comprises a central processing imit (CPU) 
72, a system controller 73, a cache memory 74, a random-access memory 75, an 
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external storage control section 76, an input control section 77, an image data 
input/output control section 78, a video controller 79, a sensor control section 80, and 
a communication control section 81. These components are connected to a bus 82. 

The external storage control section 76 serves as an interface with an external 
storage device. The section 76 is connected to, for example, a hard disk drive 83 or 
a CD-ROM drive 84. 

The input control section 77 functions as an interface with an input device. The 
section 77 is connected to, for example, a keyboard 85, a mouse-type pointing device 
86. 

The image data input/output control section 78 is provided as an interface with 
a device which handles image data. The section 78 is connected to, for example, an 
image scanner 87 or a printer 88. 

The video controller 79 serves as an interface with an image display. It is 
connected to, for example, a CRT display 89. 

The sensor control section 80 operates as an interface with extemal sensors. 
The section 80 is connected, for example, the first and second input-side sensors 12 
and 13 and the first and second output-side sensors 15 and 16, all identical to their 
counterparts of the image-processing apparatus 1 . 

The communication control section 81 functions as an interface with a 
communication apparatus. The section 81 is connected to, for example, a modem 91 
or a hub 92. The computer system 7 1 can therefore be connected to the telephone line 
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by the modem 91 that is connected to the coimnunication control section 81. 
Alternatively, the computer system 7 1 can be connected to a predetennined network 
by the hub 92 that is connected to the communication control section 8L 

In the computer system 71, the CPU 72 processes image data in the same way 
as the image-processing section 1 1 does in the image-processing apparatus 1, under 
the control of the system controller 73 with the assistance of the cache memory 74 and 
random-access memory 75, 

Hence, the computer system 71 receives the image data from the input-side 
device (e.g., image scanner 87 or the like) by the use of the image data input/output 
control section 78 or the video controller 79. In the system 71 the CPU 72 processes 
the image data in the same way as the image-processing section 1 1 does in the image- 
processing apparatus 1. The image data thus processed is output to the output-side 
device (e.g., printer 88, CRT display 89, or the like) through the image data 
input/output control section 78 or the video controller 79. 

The black-adaptation correction is effected in the computer system as in the 
image-processing apparatus 1 when the CPU 72 processes the image data, because the 
adaptation to black varies frora person to person. As a result, the colors of the images 
produced by the input- and output-side devices look almost the same, even if the 
darkest points of the input- and output-side devices differ from each other. 
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WHAT IS CLAIMED IS: 

1. An image-processing apparatus for processing the image data input 
from an image-handling device and then outputting the image data to another image- 
handling device, said apparatus comprising: 

black-adaptation correction means for correcting the image data in 
consideration of the fact that adaptation to black varies from person to person, if the 
darkest points of the image-handling devices differ from each other, so that the colors 
of the images produced by the image-handling devices look almost the same. 

2. A method of processing image data input from an image-handing 
device before the image data is output to another image-handling device, said method 
comprising the step of: 

correcting the image data in consideration of the fact that adaptation to black 
varies from person to person, if the darkest points of the image-handling devices differ 
from each other, so that the colors of the images produced by the image-handling 
devices look ahnost the same. 
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ABSTRACT OF THE DISCLOSURE 

An image-processing apparatus for processing the image data input from an 
image-handling device and then outputting the image data to another image-handling 
device. The apparatus comprises a black-adaptation correction device. The black- 
adaptation correction device corrects the image data in consideration of the fact that 
adaptation to black varies from person to person, if the darkest points of the image- 
handling devices differ from each other. Since the image is so corrected, the colors 
of the images produced by the image-handling devices look almost the same in spite 
of the fact that the darkest points of the image-handling devices differ from each other. 
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DECLARATION FOR PATENT APPLICATION (JOINT OR SOLE) 



(Under 37 CFR § 1.63; with Power of Attorney) 

FROMMER LAWRENCE & HAUG LLP flh File No. 450100-02236 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name, 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first 
and joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is 
sought on the invention ENTITLED: 

IMAGE-PROCESSING APPARATUS AND IMAGE-PROCESSING METHOD 

the specification of which 

X is attached hereto. 

was filed on as Application Serial No. , 

with amendment<s) through <if applicable, give dates). 

I hereby state that I have reviewed and understand the contents of the above- identified specification, including 
the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose to the United States Patent and Trademark Office all information known to me 
to be material to patentability as defined in Title 37, Code of Federal Regulations, Sec. 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, § 119 of any foreign application(s) 
for patent or inventor's certificate listed below and have also identified below any foreign application for patent or 
inventor's certificate having a filing date before that of the application on which priority is claimed: 

Prior Foreign Appllcation(s) [list additional applications on separate page]: Priority C laimed: 
Number: Country: Filed (Day/Month/Year): Yes No 

10-347341 Japan 7 December 1998 X 



I hereby claim the benefit under Title 35, United States Code, § 120 of any United States application(s) listed 
below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior United 
States application in the manner provided by the first paragraph of Title 35, United States Code § 112, I acknowledge the 
duty to disclose to the United States Patent and Trademark Office all information known to me to be material to 
patentability as defined in Title 37, Code of Federal Regulations, Sec. 1.56, which became available between the filing 
date of the prior application and the national or PCT international filing date of this application: 

Prior U.S. Appi 1 cation (s) [list additional applications on separate page]: 

Appln. Ser. Number: Filed (Dav/Month/Year) : Status (patented, pend ing, abandoned): 

I hereby appoint miLIAM S. FROMMER , Registration No. 25.506 . and DENNIS M. SMID . Registration No.^ 34,930 
or their duly appointed associate, my attorneys, with full power of substitution and revocation, to prosecute this 
application, to make alterations and amendments therein, to file continuation and divisional applications thereof, to 
receive the Patent, and to transact all business in the Patent and Trademark Office and in the Courts in connection 
therewith, and specify that all communications about the application are to be directed to the following correspondence 
address: 

Esq. Direct all telephone calls to: 

(212) 588-0800 
to the attention of: 

WILLIAM S. FROMMER 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on 
information and belief are believed to be true; and further that these statements were made with the knowledge that 
willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of 
Title 18 of the United States Code and that such willful false statements may jeopardize the validity of the application 
or any patent issued thereon. 

INVENTOR(S): 
Signature: 



WILLIAM S. FROMMER 
c/o FROMMER LAWRENCE & HAUG LLP 
745 Fifth Avenue 
New York, New York 10151 



Date: 



Full name of sole or first inventor: 
Residence: 
Citizenship: 



Kiyotaka NAKABAYASHI 
Tokyo, Japan 
Japan 



Signature: — • 

Full name of 2nd joint inventor (if any): Naoya KATO 
Residence: Chiba, Japan 

Citizenship: Japan 



Signature: 

Full name of 3rd joint inventor (if any): 
Res i dence : 
Ci tizenship: 

[Similarly List additional inventors on separate page] 

Post Office Address (es) of inventor(s): 

[if all inventors have the same post office address] 



Date: 



Date: 



Sony Corporation 
7-35 Kitashinagawa 6-chome 
Shinagawa-Ku, Tokyo 141, Japan 



Note: In order to qualify for reduced fees available to Small Entities, each inventor and any other individual or entity 
having rights to the invention must also sign an appropriate separate "Verified Statement (Declaration) Claiming [or 
Supporting a Claim by Another for] Small Entity Status" form [e.g. for Independent Inventor, Small Business Concern, 
Nonprofit Organization, individual Non- Invent or] . 

Note: A post office address must be provided for each inventor. 
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